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Abstract 

From an improvement research perspective, this study explores strategies used to implement a school-wide vocabulary 
intervention into language arts instruction at an urban elementary school. Academic language time, an innovative 
change in the instructional delivery system, allots time and structure for deliberate teaching of cross-disciplinary 
academic vocabulary as a routine practice in the language arts block. Implementation included a set of core strategies 
delivered by a university-school purveyor team. Implementation evidence consisting of survey, observation, and 
interview data was collected over one school year. Quantitative and qualitative analytic techniques were applied to 
determine effectiveness of the implementation strategies. Results identify strategy characteristics that supported or 
impeded the implementation process, the valence of strategy patterns, and areas for improvement. The strategy analysis 
provides fine-grained descriptive information that informs the specification and description of implementation strategies 
for purposes of innovation and change. 
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1. Introduction 

The literacy field has gained considerable ground as to what works for improving students’ reading skills. National 
reports provide scientific evidence of best practices in reading (National Early Literacy Panel, 2008; National Reading 
Panel, 2000) and repositories offer evidence-based practice guides for classroom instruction and vetted interventions 
(e.g., Reading Rockets http://www.readingrockets.org/). Knowledge of what works, however, has not easily transitioned 
into classroom practice. What we need, some argue, is not more evidence of what works, but rather research that “takes 
what we know and turns it into what we do” (Peters, Tran, & Adam, 2013). In some fields, this is referred to as 
implementation science, which examines the components of change for improved practices in local settings (Fixsen, 
Naoom, Blase, Friedman, & Wallace, 2005) and, in others, as improvement science, which iteratively tests 
implementations to improve practical guidance for evidence-based practice (Bryk, 2015; Rubenstein & Pugh, 2006). 

In the panoply of factors that influence the implementation process, we focus on implementation strategies—what they 
are, how they work, their strengths and vulnerabilities. It is at the strategy level where effective implementation is 
ultimately achieved. Implementation models (e.g., the Logic Model [Hernandez & Hodges, 2003]) provide frameworks 
for guiding the implementation process and include information about using strategies at the level of principles and 
generalizations. Few, however, delineate strategies sufficiently to offer strategic direction and action “on the ground.” 
Yet it is at the ground level where even the best-planned strategies become vulnerable to local setting conditions for 
better or worse. And it is at this intersect where we fix our attention on the larger goal of helping to improve the 
implementation process. 

In this study we explore the implementation of evidence-based vocabulary instruction in K-5 classrooms at an urban 
elementary school. We make the assumption that if the vocabulary instructional techniques are research based and 
implemented routinely with some measure of fidelity, we should see improvement in students’ academic word learning. 
This makes sense. The linearity of the logic, however, is beset with complexities as the growing body of 
implementation research has shown (Fixsen, et al., 2005). 

1.1 Implementation Strategies 

1.1.1 Definition 

In the field of implementation science, implementation strategies are critically important because they constitute the 


184 



Journal of Education and Training Studies 


Vol. 5, No. 5; May 2017 


how to component of the change process. Defined as the means or methods for adopting, implementing and sustaining 
interventions, implementation strategies are necessary for realizing evidence-based instruction in classrooms. 
Well-formulated strategies can quicken the pace and effectiveness of the implementation process. Research evidence of 
effective implementation strategies in implementing educational interventions, however, is sparse, due partially to a lack 
of common language and a common framework for thinking about implementation (Proctor, Powell, & McMillen, 
2013). 

1.1.2 Research Frameworks 

In a comprehensive research review of the influence of the implementation process on program outcomes, Durlak and 
DuPre (2008) found that “the level of implementation achieved is an important determinant of [intervention] outcomes,” 
increasing program success while also providing stronger benefits for participants” (p. 334). Using an ecological 
framework of effective implementation, they hypothesized that implementation is influenced by factors present in five 
categories: (a) innovations, (b) providers, (c) communities, (d) delivery system, and (e) support system. A number of 
strategies are embedded within these categories. For example under the heading specific practice and processes of the 
delivery system, the authors list shared decision-making, coordination with other agencies, frequent and open 
communication, and the formulation of tasks. Based on their research review, shared decision-making (e.g., local 
collaboration), consistently led to better implementation. Moreover other data showed that shared decision-making 
predicted program sustainability. Shared decision-making, therefore, emerged as a potent strategy, although its 
operationalization remains sketchy (e.g., the “how to” of collaboration). 

In a broad sweep of the literature, Proctor et al. (2013) characterized the type, range and nature of implementation 
strategies, grouping them as “ ‘top down/bottom up,’ ‘push/pull,’ and ‘carrot/stick’ tactics, mainly involving ‘package’ 
approaches” (p. 2). Strategies consist of training methods, tool kits, checklists, protocols, and action cycles. Their 
complexity varies widely from a single strategy (e.g., dissemination of best practice guidelines) to an array of strategies. 
Proctor et al. (2013) for instance, assembled a menu of 68 implementation strategies grouped into six processes: 
planning, educating, financing, restructuring, managing, and attending to the policy context. These processes included 
familiar strategies of educational intervention, i.e., conducting a needs assessment, developing an implementation plan, 
providing feedback, etc. For purposes of manageability, though, strategies are most often combined to form a 
multifaceted strategy, such as train-coach-observe-give feedback, or have been packaged and branded, such as 
Plan-Do-Study-Act. 

Some of the strongest research on implementation strategies has provided evidence on what does not work. 
Experimental studies show, for example, that the dissemination of information strategy does not impact practitioners’ 
performance or improve intervention outcomes (Azocar, Cuffel, Goldman, & McCulloch, 2001; Azocar, Cuffel, 
Goldman, & McCarter, 2003). Comparing a single mass mailing of evidence-based guidelines or a mass mailing plus a 
client-targeted letter with a control, no difference was found on practitioner behavior in a mental health context. 
Relatedly, a random assignment of schools implementing a skin care health program to either a mailed materials 
condition or a mailed materials + staff development module condition showed no differences in the sun protection 
instructional practices in either group (Schofield, Edwards, & Pearce, 1997). Educational research generally converges 
on the finding that training alone through single or multiple-series professional development (PD) workshops 
disseminating best-practice information does not result in positive implementation outcomes (Darling-Hammond, Wei, 
Andree, Richardson, & Orphanos, 2009; Garet, Porter, Desimone, Birman, & Yoon, 2001; Webster-Wright, 2009). 

A few experimental studies provide evidence of strategies that do work to good effect. For instance, Biancarosa, Bryk, & 
Dexter (2010) argue that sustained training of literacy coaches with feedback from program developers has a positive 
impact on teaching, which, in turn, improves student literacy learning. In their four-year longitudinal field trial studying, 
the effects of Literacy Collaborative (LC), 16% of students made literacy gains as compared to baseline data in year one, in 
year two 28% and by year three 32% showed literacy growth. In contrast to other studies involving literacy coaches 
(Marsh et al., 2008), LC coaches received a full year of professional development before beginning to work with teachers. 

1.1.3 Research Directions 

In a cross-disciplinary review of experimental studies, Fixsen et al. (2005) identified four broad themes relevant to 
implementation strategies. One, guidelines alone or practitioner training alone are not effective. The Early Reading 
Professional Development Interventions study in 90 high-poverty schools, for example, tested the effectiveness of two 
PD interventions (random assignment) over one-year, reporting mixed results in improving teachers’ knowledge and 
practice and students’ reading achievement (Garet, et al., 2008). Teachers in Intervention A participated in an eight-day 
institute and seminar series, those in Intervention B in the institute series plus twice-a-week coaching, and those in a 
control condition participated in district-provided PD per usual. Although the PD interventions resulted in positive 
impact on teachers’ knowledge and their fidelity to a target instructional practice compared to the control, there were no 
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statistically significant differences in achievement between students in the intervention and the control schools. Further, 
there were no statistically significant differences between the PD intervention groups on teacher knowledge and 
practices in the implementation year. Thus, short-term training, even with follow through in-classroom support, did not 
achieve the intended outcomes of the intervention. 

Two, longer-term, multilevel implementation strategies, however, are more effective. A high level of involvement by 
program developers is a core feature of active approaches to implementation likely to be effective in implementation. 
This is borne out in Goldenberg’s study of a Los Angeles elementary school through a five-year development period 
(Goldenberg, 2004). Grounded in Tharp and Gallimore’s theory of assisted performance (Tharp & Gallimore, 1988), 
Goldenberg details the activity settings that support and sustain the school’s and community’s activities and strategies to 
achieve their shared goal of improving the literacy achievement of the largely Latino school population. Careful, 
collaborative planning and strategic implementation was key to achieving the intended outcomes of the initiative. 

Three, analyses of practitioner selection and training provide guidance for bringing effective implementation to new 
settings. Criteria for selecting practitioners who will be involved in the implementation process are helpful, although the 
actual selection process, fidelity issues, and variability of participants can heavily influence adherence to criteria. Whole 
school-site implementation, for example, includes all the teachers whether they meet criteria of participation or not. An 
understanding of what constitutes the crafting of selection and training in situ is limited and more systematic analyses 
are needed to improve these strategies (Penuel, Allen, Farrall, & Cobum, 2015). 

Four, not enough is known about the functioning pieces and parts of different implementation factors (e.g., coaching or 
fidelity) and how these work (or not) within and across sites. To this need, Proctor et al., (2013) argue for the better 
specification and reporting of implementation strategies in implementation research. They discuss fundamental 
principles of naming, defining, and specifying implementation strategies when setting out to study them in context. 
They advise that naming strategies should be grounded in prior research to the extent possible; that specific strategies 
should be defined conceptually as to what they involve; and that strategies should be operationalized as to who does 
what, when, and to what end or outcome. 

1.1.4 Vocabulary Intervention 

We focus on implementation strategies related to vocabulary intervention in classroom settings for two reasons. One, 
vocabulary is a vital link between word reading skills and higher-level comprehension, and vocabulary size is a strong 
predictor of reading comprehension after Grade 2 (Perfetti & Stafura, 2014; Scarborough, 2001; Storch & Whitehurst, 
2002). Vocabulary is also the gateway to academic content knowledge, conveying the central ideas, concepts and 
processes of a discipline (Bloom, 2002). Two, while stronger vocabulary instmction is needed, it is still quite difficult to 
implement in routine practice. Despite voluminous literature on the effectiveness of many vocabulary strategies (Butler 
et al., 2010) few studies have looked at comprehensive vocabulary instmction that can be feasibly implemented by 
classroom teachers (Pressley, Disney, & Anderson, 2007). Fewer still identify, define and operationalize the 
implementation strategies used in the implementation process. 

Case example: in a study examining the implementation of an academic vocabulary intervention with linguistically 
diverse students in urban middle schools, Lesaux, Kieffer, Faller, & Kelley (2010) found promise, yet challenges related 
to ease of implementation in teachers’ practice. An instmctional cycle was followed for the weekly instmction and 
teachers met with a program specialist monthly. Analyses revealed not only the challenge of meeting student need and 
the importance of appealing text, but also the need to build teachers’ instmctional capacities. Relatedly, Baumann et al. 
(2013) implemented a comprehensive vocabulary instmction program with middle grade students. Over a three-year 
design research study, researchers developed and evaluated a multifaceted, comprehensive vocabulary instmction 
program (MCVIP). Successful implementation took a full three years before significant vocabulary gains were realized. 
Issues ranged from teachers’ frustration with lack of time, perceptions that students already knew selected words, 
inability to identify and teach academic vocabulary, disconnect from the curriculum, and frustration with university 
researchers. Results highlighted the need to develop more intense professional development to sustain and support 
practitioners, especially in the initial year of implementation. It also pointed to the urgent need to move beyond 
focusing only on intervention effectiveness to describing ‘how to’ instantiate ongoing collaboration with teachers. 

2. Research Questions 

Our central goal in this study was to identify (name), define, and examine the operationalization of strategies used to 
implement a multi-faceted vocabulary intervention, referred to as Academic Language Time (ALT) into the language 
arts block in elementary classrooms. Based on this descriptive information, we examined the effectiveness of the 
strategies in context, observing positive and negative patterns of strategy use. From this case example we make 
inferences and draw conclusions toward the larger goal of explaining how implementation strategies can work for or 
against desired change in a setting. Our study was framed by the following questions. 
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(1) What strategies were used and by whom? 

(2) How were the strategies operationalized in the context? How did they work? 

(3) What patterns emerged that supported or impeded the implementation process? 

(4) How effective were the strategies for implementing the intervention? 

3. Method 

We used a descriptive research design to qualitatively analyze the implementation strategies of the ALT intervention, 
triangulating data across several sources, including teacher surveys, observation protocols, and interview transcriptions. 
Our analyses focused on (a) identifying the strategies and their purveyors, defined as those individuals or groups who 
actively worked to implement the intervention (Fixsen et ah, 2005); (b) describing how the strategies were 
operationalized, i.e., put into play in the context; (c) identifying patterns of strategy use that supported or impeded 
implementation; and (d) determining the relative effectiveness of the strategies for implementing the intervention. 

3.1 Context 

The study was conducted at an urban elementary STEM school, serving 258 students pre-kindergarten through grade 5, 
most residing outside of the school’s immediate neighborhood. It was embedded in a five-year research agenda involving 
the replicability and sustainability of a structural change in language arts instruction referred to as the DigiLit Project. 
DigiLit introduced a mobile learning environment (1:1 iPad deployment), direct instruction of academic vocabulary and 
structured independent reading time into the language arts block as mechanisms for strengthening teaching practice. 

The student population was majority African American (76%) followed by White (8%), Hispanic (6%) and multi-racial 
(1%) groups with 10% of the population identified for special education services; 67% were classified as economically 
disadvantaged. The study followed the implementation of the vocabulary intervention school-wide over one school year, 
involving 14 classrooms and 18 teachers. Implementation occurred over three cycles, each about three months in length. 
Full implementation occurred in cycles 2-3 for approximately 16 weeks, totaling an estimated 120-125 hours of 
language arts instruction. 

3.2 Intervention 

Academic Language Time (ALT) introduces 15-20 minute daily lessons devoted to direct instruction in disciplinary 
content vocabularies. In this school site, the focus was on STEM vocabulary. Each vocabulary lesson motivates word 
learning; teaches 3-5 new words using a menu of evidence-based techniques; asks students to use/read new words in 
context; and explores new words in small group activities (Beck, McKeown, & Kucan, 2002; Biemiller & Boote, 2006; 
Stahl, 2003). The direct instruction follows a protocol where teachers ask students to (a) pronounce new words; (b) turn 
and talk to define words with each other; (c) listen for/read new words in context; and (d) explore new words in small 
group activities. Each ALT lesson is fast-paced, providing multiple exposures to new content words. The intervention 
promotes the use of digital devices, such as iPads, and electronic resources, such as e-texts, e-books and apps, to 
support student practice with content words. On an intervention continuum, ALT is considered a mild to moderate 
intervention. Its instructional delivery is at Tier 1 by the classroom teacher who has received training in direct 
vocabulary instruction procedures and systematic progress monitoring of target word learning (Glover, 2010). 

3.3 Data Sources 


Descriptive evidence of the implementation strategies was drawn from several data sources: (a) syntheses of the 
literature to identify strategy terms and definitions (Durlak & DuPre, 2008; Fixsen, et al., 2005; Proctor et al., 2013); (b) 
pre/post surveys and interview data to describe the operationalization of strategies; and (c) observational and interview 
data to describe strategy effectiveness. Triangulating across data sources provided a multi-layered view of strategy 
implementation. (See Figure 1.) 


Data Source _ 

Professional Development 
Content 

Professional Development 
(PD) Satisfaction Survey 
Intervention Observation 
Form 

TPACK Survey 


Exit Interviews 


Description _ 

Knowledge and skills essential for implementing the intervention 

Online survey consisting of eight items that rated level of 
satisfaction on a 5-point scale + open-ended comments 
A form consisting of 22 criteria that rated adherence to the AFT 
protocol per lesson segment on a 4-point scale. 

A 53-item questionnaire that measured teachers' perceived abilities 
in integrating technology into their teaching on a 5-point scale 
(Hosseini & Kamal, 2012) 

A structured interview consisting of 9 questions about the 

intervention and its implementation _ 

Figure 1. Data sources, description and collection. 


Collection _ 

5 full-day sessions conducted 
monthly; 1 half-day session 
Post each session; 6 per 
teacher 

6 per teacher 

Pre/post implementation 


1 per teacher; n=18 
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3.4 Data Analysis 

Descriptive statistics were used to analyze survey and observational data. Mean ratings were calculated for each item of 
the PD survey and in total. TPACK pre-post responses were analyzed using a paired-sample t- test comparison of mean 
teacher ratings to determine change in perceptions of technology integration knowledge in seven knowledge categories. 

Observational data were analyzed for fidelity to the ALT instructional protocol. Fidelity was calculated for both extent 
and quality of adherence using the grade level as the unit of analysis because teachers co-planned instruction and shared 
students. The number of observed criteria per lesson segment, divided by the total number of criteria, yielded a 
percentage for extent of fidelity by grade level. The quality was calculated from the individual ratings of each criterion 
in each lesson segment. Individual criterion ratings were summed to calculate a mean score by segment and a lesson 
segment total. Mean scores were then aggregated to determine degree of fidelity to each lesson segment by grade level. 

Interview transcripts were analyzed using ATLAS.ti software (ATLAS.ti GmbH, 2013). Transcripts were segmented 
into text quotations (topic-related segments) and coded for evidence of strategy components (“how to” statements). The 
research team initially coded the corpus for strategy statements, achieving consensus through discussion. Next the team 
coded the statements for strategy type and definitional terms (n=314), achieving inter-coder reliability on statements 
from two interviews (92%). Given the volume of individual word descriptors contained in the statements (-4500 words), 
a random sample (n=211) of statements from the total pool of statements was coded for word descriptors of salient 
features. Since the interviews probed teachers’ perceptions of the successes and challenges of the intervention and its 
implementation, the research team also coded the statements for valence (positive [+] or negative [-]) related to 
strategies. 

Analytic induction was used to triangulate across data sources to identify strategy categories and relationships between 
categories and to develop analytic frameworks that described the strategies used in the setting, emerging patterns, and 
strategy effectiveness (Goetz & LeCompte, 1984). 

4. Results 

4.1 Strategies Used and Purveyors 

Repeated readings of PD and interview data yielded evidence of four purveyor groups who each used a core strategy to 
put the intervention in place school-wide. A leadership team consisting of university partners, a district grants 
administrator, the school principal and the literacy lead teacher led the implementation. The group met regularly to plan, 
launch and monitor the intervention. Their core strategy was one of coordinating, defined as cooperation and 
collaboration among the members to bring different perspectives, skills and resources to bear on the implementation. 
Notable here is the composition of the team, which included university, school district, building level and teacher 
representation, thus addressing one of the key criteria of an effective implementation strategy supported by research 
(see, for example, Durlak & DuPre, 2008, Organizational Capacity-Specific Practices and Processes chart, p. 337). 

Grade level teams met at monthly professional development sessions (-2 hours for planning) and weekly at 
Teacher-Based Team meetings (TBTs) (-45 minutes) to plan the intervention, including: (a) selecting target words for 
instruction; (b) creating word cards/charts; (c) selecting print and digital activities for vocabulary practice; (d) 
developing formative assessments of student word learning; and (e) reviewing student performance data for 
instructional decision making. Their predominating strategy was one of creating content for the intervention and 
entailed planning for instruction, preparing materials and considering the “fit” of the intervention in routine practice. 
Often referred to broadly in the literature as “activities” belies the considerable effort, complexity and attention to detail 
that was required. 

A professional development team led by university faculty in collaboration with the principal and lead literacy teacher 
developed and delivered the content of the monthly professional development (PD). Focusing more on “how to” than 
advancing teachers’ pedagogic cognitive development (e.g., reflection), they employed what is probably best termed a 
training strategy, which is frequently described in the literature and regularly used in in the implementation process. In 
this setting, the strategy embraced a high level of teacher participation in the content. Teachers shared their expertise on 
app usage for instruction and taught one another technology skills; they problem solved technology, classroom 
management and instructional issues; they developed mechanisms for data management and sharing (e.g. Google docs); 
and they critiqued the intervention design, suggesting adjustments and adaptations that made it more workable under 
classroom conditions. 

An app team that included a small group of tech-savvy classroom teachers assumed responsibility for the management 
and usage of mobile devices and educational apps at the building level and provided support to classroom teachers. The 
main strategy used by the group was one of troubleshooting app usage to promote and support the digital option of the 
intervention in classrooms through resources such as re-training, emotional support, and ways to solve problems at hand. 
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Considerable implementation research, especially in the use of classroom technology, indicates the critical importance 
of just-in-time technical support as a key strategy. 

4.2 Strategies Operationalized 

Our second pass through the data sources further parsed the strategy types to identify their salient features and to gauge 
their valence as expressed by the participants. We conducted content analyses of a random sample of quotations from 
the interview data to determine strategy features and valence. We excluded survey data in this analysis and focused on 
interview date because these data related the strategy experience in the participants’ own words. We viewed valence 
through a psychological lens, observing the attractiveness (positive) or averseness (negative) to a strategy. We took this 
perspective because we were interested in what the participants felt and thought about the implementers, actions, targets 
and intensity of the strategies, reasoning that this approach might provide insights into how well the strategies worked. 
Strategy effectiveness that results in acceptability and adoption of interventions is multi-faceted, and involves winning 
both hearts and minds (Proctor et al 2013). The results of this fine-grained analysis are summarized in Table 1. 

Table 1. Frequency of Strategies by Type, Features and Valence 


Core Strategy 


Features 


Valence 

Type 

Total 

Feature key word 

Total 

+ 

- 

Coordination (CO) 

57 

COl sharing 

17 

6 

11 



C02 directing 

12 

4 

8 



C03 prioritizing 

17 

6 

11 



C04 focusing 

11 

7 

4 

Creating Content (CC) 

91 

CC1 collaborating 

7 

7 

0 



CC2 deciding 

8 

7 

1 



CC3 evaluating 

47 

38 

9 



CC4 adapting 

29 

18 

11 

Training (TN) 

42 

TNI modeling 

17 

12 

5 



TN2 practicing 

13 

9 

4 



TN3 observing 

4 

0 

4 



TN4 feedback 

8 

1 

7 

Troubleshooting (TB) 

21 

TB1 teaching 

0 

0 

0 



TB2 demonstrating 

1 

0 

1 



TB3 technical 

17 

2 

15 



TB4 managing 

3 

0 

3 

Total 

211 


211 

117 

94 


The results highlight the predominance of the Creating Content strategy in the implementation process. Its main features 
included evaluating the compatibility of the intervention (“fit” with teachers’ instructional priorities and values), and 
adapting it (modifying it for better fit in the instructional context). For the most part, teachers perceived these features 
of the strategy positively, which tended to support implementation at the classroom level. For example: addition of the 
technology (iPads) “changed [ALT] ...adding in that tech piece of it, you know trying to add that in and change how you 
know...we have read more books, digital books outside of what we were doing [before]” (Ql:07 [Quotation 1, 
Participant 07]; CC4-adapting). 

Majority features of the Coordination strategy included sharing information about the intervention and prioritizing tasks 
for implementation, and both were perceived negatively by the teachers, and thus impeded classroom implementation. 
The weak formulation of the ALT task, for instance, created more work for the teacher: “Running, you know, make the 
cards, and find the stories and all that kind of thing, and it, I kept thinking it's going to become routine, it's become part 
of the process, but it never quite did” (Q3:16; C03: prioritizing). And, the lack of procedure and effective time 
management frustrated the teachers in implementing the ALT intervention: 

I did see a bit of a jump in the stress level on the teachers though because it was compared to what we did last in 
piloting at least the vocabulary protocol, just trying to source the books you were going to use, or the [ALT] books 
you were going to use, or to figure out the vocabulary you wanted to teach especially when the standards tell you not 
to go in a certain direction because they ’re going to get that in the next grade level. It was very stressful that way 
trying to find twenty words per week at first and then to have it change to twenty words over ten days (Q10:32; COl: 
sharing). 

Modeling and practicing features marked the Training strategy. Teachers, for example, commented most frequently on 
the benefits of showing them how to use an app or how to implement the vocabulary procedure along with the 
opportunity to practice using apps in vocabulary teaching. These features were generally perceived favorably, thus 
contributing to the overall effectiveness of the implementation. As one teacher explained, “ ...the thing that I really liked 
is when teachers shared some of the apps [for ALT] that they were using that I could take back and use in the classroom. 
I thought that was very helpful” (Q7:18; 77V-7: modeling). Although fewer in number, the majority of quotations 
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directed to classroom observations and receiving feedback on instruction were perceived negatively, which, in turn, 
decreased the effectiveness of this strategy. For example: 

It was just sort of disconnected for us, I felt and it probably was for them too so when we ’d hear feedback it was kind 
of uh, and you know, and the feedback wasn ’t always the most, it wasn’t that constructive and we’d like to hear if 
they ’re going to observe to do to find out what we ’re doing well and not well, then we ’d like to hear that and get more 
feedback or just not bother coming in (Q5:7: TN-4: feedback). 

Troubleshooting was the least-used strategy, limited primarily to addressing and managing digital technical issues (e.g., 
iPad screens freezing) efficiently and effectively. Comments regarding how to use the technology tools, operate apps or 
deal with the issues that occurred (adding/removing apps, glitches in access) indicated that these issues were not 
routinely addressed and that overall tech support was perceived negatively. Clearly this strategy was not working. 
Comments along the lines of the following were common: 

...some of the apps were really useful but one of the downfalls we have been discussing is we only have like the free 
versions the kids can only use them like one time... one of the apps kept glitching and we weren ’t able to use it like we 
did like last year we were to have the full function of it, um some of the kids got frustrated because it would freeze up 
on them um with Storia some of it was very hard for them to log in cause the passwords that they gave the kids it was 
way too long and it was hard for them to manipulate (Q7:16; TB-3: technical). 

In sum, the Creating Content strategy in conjunction with the Training strategy favored the implementation of the ALT 
intervention. Teachers felt it was beneficial to the students. They were willing to try it and perceived it as relevant to their 
instructional goals in the language arts. They generally found the professional development to be helpful, especially in 
showing them how apps and procedures worked and providing time for collaborative planning, although there was a 
strongly expressed need for more observation and feedback on how well they were delivering the implementation at the 
classroom level. 

On the flip side, the teachers lamented the lack of coordination in communicating expectations, formulating tasks and 
roles, setting priorities and managing the overall implementation process. This was especially acute with respect to 
Troubleshooting, which failed as a strategy. The negativity assigned to these strategies points to insufficient guidance and 
support at the level of detail and individual ‘know how’ needed to implement the intervention successfully. 

4.3 Strategy Patterns 

Our next analysis searched the interview data for patterns that described how the strategies interacted to influence 
implementation. At a broad level the overall valence of the Creating Curriculum (CC) and Training (TN) strategies was 
generally positive while that of Coordinating (CO) and Troubleshooting (TB) was negative. The CC strategy was the 
most positive and the TB strategy was the most negative. The TN strategy is about even between positive and negative 
perceptions of effectiveness. 

A more fine-grained analysis identified salient features contributing to positivity or negativity. Figure 2 illustrates the 
negative perceptions of the sharing and directing features of the Coordinating strategy combined with the 
demonstrating, technical and managing features of Troubleshooting, which suggests-weaknesses at the leadership level 
that may have compromised implementation at the classroom level. The semantic network between CO and TB 
provides evidence of organizational features (e.g., lack of integration) and practices (e.g., weak technical assistance) 
that discouraged implementation. Commenting on the technical function of the troubleshooting feature, a teacher stated: 
"I think that some of the technology they want us to use I mean I it didn ’t work so that was a big you know deterrent for us 
so there” (Q5:14: TB-3: technical). 
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TB4 managing 
TB3 technical 
TB2 demonstrating 
TB1 teaching 
C04 focusing 
C03 prioritizing 
C02 directing 
COl sharing 

0% 20% 40% 60% 80% 100% 































■ negative 

■ positive 


Figure 2. Percentage valence of the Coordinating-Troubleshooting features 

Figure 3 displays the CC-TN pattern which trends positively with the exception of classroom-based observation and 
feedback. The generally positive features of collaborating, deciding, evaluating, adapting and modeling combined to 
create supportive strategies that helped to make the intervention work at some level of effectiveness. The semantic 
network indicates some attention to the characteristics of ALT (e.g., using apps to rehearse new words) and skill 
acquisition (e.g., modeling) needed for its successful implementation, but not enough attention to performance feedback. 
The following quotations highlight these “enhancing” strategy links: 

Evaluating apps for Creating Content: They [apps] were very motivating for students especially things like Popplet for 
getting their ideas organized, then to write from that so they could use it for research purposes. I liked some of the 
other ones we had like the Write About app was good (Q10:21; CC-3: evaluating). 

and modeling linked to Training: ‘Especially when the staff would share what apps they were using, how they were 
using it and having time to work with our team you know to plan different things and implement those things with our 
class ” (Ql:04; TN-1: modeling; CC-1: collaborating). 


TN4 feedback 
TN3 observing 
TN2 practicing 
TNI modeling 
CC4 adapting 
CC3 evaluating 
CC2 deciding 
CC1 collaborating 



■ positive 

■ negative 


0% 20% 40% 60% 80% 100% 


Figure 3. Percentage valence of the Creating Curriculum-Training features 

Figure 4 depicts the positive and negative strategy combinations, revealing proportionally more negative than positive 
feelings toward the implementation. The synergy of the CO-TB strategy features (average of 65%) appeared to 
overwhelm the combo of CC-TN strategy features (average of 55%). The resulting 10% difference (trending negative) 
in the core strategy pairs may have been enough to influence the overall implementation process. 
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■ positive ■ negative 


Figure 4. Valence proportions of strategy pairs by feature 


4.4 Strategy Effectiveness 

Analyses of the PD survey data provide another perspective of strategy perception and effectiveness. The survey 
included eight items related to overall satisfaction and with specific features such as handouts, practice time, presenter 
effectiveness, collaborative opportunities, and usefulness of content and strategies. The results of aggregated teacher 
feedback across five PD sessions show the mean ratings of teacher satisfaction on a 5-point scale (5 = strongly agree; 4 
= agree; 3 = neutral; 2 = disagree; 1 = strongly disagree). The mean average of nearly all ratings were in the 4.00 to 4.76 
range (agree), indicating a generally positive response to PD content and process. 

The TPACK analysis provides additional feedback on the teachers’ perceived effectiveness of the training strategy. A 
comparison of teachers’ pre/post responses to the questionnaire indicates significant mean differences (p<.005) in their 
perceived knowledge in all seven categories before and after implementation. Teachers clearly perceived that they knew 
more in these knowledge domains and how to integrate their tech knowledge into implementing the intervention. 

The observational data, however, paint a more negative picture of training effectiveness. (See Table 2) Overall the 
extent of implementation across the grades fell below 60%, with PK and G1 closest to the benchmark (60%) and G2 
and G4 the most distant. Adherence to the motivate and teach segments of the lesson structure was strongest and that to 
the explore segment the weakest. 

The quality of implementation was more favorable with respect to Training, with overall averages ranging from 3.3-3.8 
across grades, and a total mean of 3.6, indicating a moderate degree of adherence to segment-specific criteria when a 
segment was actually implemented. Most of the grades showed moderate to high adherence with G2 the weakest, 
although G2 teachers may have made more adaptations to the intervention as suggested in the qualitative analysis. For 
example: 

If I'm in a small group, I almost prefer and had the best success in using the iPads with me in a small group because 
then I can control exactly what they ’re doing and they ’re not getting off task, and then I would leave the iPad use for 
independent reading (Q9:3; CC-4: adapting). 

Table 2. Extent and Quality of Fidelity to the Academic Language Time (ALT) Intervention 


Extent of Fidelity by Segment (%)Quality of Fidelity by Segment (mean rating 1-4) 



Obs 

Tchrs 

Motivate 

Teach 

Read 

Explore 

Total 

Motivate 

Teach 

Read 

Explore 

Total 

PK 

10 

2 

54 

91 

72 

14 

57.8 

3.2 

3.8 

2.8 

3.6 

3.35 

K 

11 

2 

42 

84 

66 

29 

55.3 

2.6 

3.6 

2.9 

3.2 

3.08 

G1 

13 

2 

100 

59 

41 

29 

57.3 

3.9 

4.0 

3.9 

3.7 

3.88 

G2 

12 

2 

58 

32 

26 

26 

35.5 

3.7 

3.8 

3.3 

2.9 

3.43 

G3 

16 

4 

56 

66 

43 

24 

47.3 

3.4 

4.0 

3.7 

3.7 

3.7 

G4 

7 

2 

60 

52 

6 

18 

34 

3.0 

3.4 

2.3 

3.7 

3.1 

G5 

9 

2 

47 

89 

29 

19 

46 

3.1 

3.9 

3.5 

3.9 

3.6 

Avg 



59.6 

67.6 

40.4 

22.7 

47.6 

3.27 

3.79 

3.2 

3.5 

3.6 
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From these results, we learn that by and large the implementation strategies failed to support implementation fidelity 
across classrooms, ranging from a low of 34% (G4) to a high of 58% (PK). The early primary grades demonstrated 
stronger fidelity than G2-G5, which suggests potential differences in the training at the PK - G2 and G3 - G5 grade 
levels. Still, the strategies were too weak to integrate the ALT structure into routine practice, although the degree of 
fidelity on the Teach segment of the ALT lesson framework appeared strong (3.4 - 4.0 range; average 3.79), perhaps 
due to the teachers’ familiarity with basic teaching techniques. 

5. Discussion 

Our study explored the effectiveness of implementation strategies used to embed a vocabulary intervention in routine 
language arts instruction at an elementary school. We wanted to name, define and describe how a set of strategies 
worked in the local setting to better understand what happens when strategies are put into action on the ground, i.e., 
situated. This kind of information helps to specify strategy features and inform tactics for improving the implementation 
process. Based on participant feedback and behaviors, we observed the valence (positive or negative qualities) 
attributed to strategy clusters that worked for or against effective implementation. Negative perceptions of even well 
formulated strategies, we observed, may overwhelm and compromise the fidelity of implementation. 

These results are not surprising, corroborating what has been observed in prior implementation research. 
Troubleshooting technology, for example, was the least effective strategy, largely because of insufficient support at the 
building level. The coordinating strategy also lacked communication and detailed implementation guidance—problems 
often cited in the implementation literature. Considerable research demonstrates the critical importance of effective 
mechanisms that encourage frequent and open communication yet “failures to communicate” persist in the adoption of 
innovations. Relatedly, the need for specific, detailed guidance to scaffold the ‘new’ into routine practice cannot be 
overemphasized. Training, here, also fell short in skill-building areas, such as: demonstrating how the intervention 
actually should work in the classroom; providing sufficient curricular resources and co-planning time; differentiating 
PD to meet personal needs; and advancing teachers’ instructional and technology skills in the instructional delivery. 
Together these known pitfalls of implementation that surfaced in this effort support a major finding from research, 
namely that threshold conditions should be met to increase chances of success (Fixsen, et al., 2005). 

We also discovered some unknowns in relation to what we already know about the vulnerabilities of frequently used 
strategies. Teachers’ knowledge and skills in creating curriculum, for example, emerged as a critical factor of efficient 
and effective implementation of the vocabulary intervention. Although our participants were experienced educators at 
teaching reading and relatively confident in their technology skills, they struggled with creating activities and materials 
for effective vocabulary instruction. We cannot account for this only to speculate that it may be due to the “packaged” 
nature of most language arts curriculums where instructional content is manualized (what; how) and delivered by the 
teacher (who). As a result educators do not have much practice in actually creating curriculum, and thus do not often 
exercise the pedagogic knowledge and skills of curriculum design. So the cognitive demands of this professional work 
seem hard and time-consuming as was frequently expressed by these teachers. 

Certain features of collaboration, initially invisible to us, also emerged in the implementation process. While 
collaboration “worked” to establish the setting for implementation (e.g., a technology infrastructure), the 
implementation effort lacked the know-how for sustained engagement with the intervention. Unknown to us at the time 
was the deep need for shared expertise that generated a common knowledge of the intervention goals, processes and 
potential outcomes. Shared expertise combines the unique expertise of researchers and teachers to bridge practice 
boundaries for rich knowledge exchange that yields common knowledge (Edwards, 2012). It involves the ability for 
stakeholders to recognize the standpoint of others and to be explicit and comprehensible when discussing their own 
interpretations and ways of responding to the problem they are working to solve. Unfortunately our implementation 
process did not create this kind of deep learning space that might have made the intervention better understood both 
from the standpoint of the researchers who had the idea and teachers who attempted to translate that idea into action. 
This need for shared expertise unveils a hidden feature of collaboration not often discussed: that experts—researchers 
and practitioners—must actively extend their knowledge to build links and try to integrate what they know with what 
others want to, or should know and do (Nowotny, 2003, p. 155). 

The results of our study are helpful on two counts. One they contribute to the need for better specification of particular 
implementation strategies found to impact the implementation process in professional practice. They add detail to the 
definition and description of common implementation strategies, and they describe how the strategies were 
operationalized in the setting. This kind of information helps to build the groundwork for replication that strengthens 
strategy specifications both conceptually and operationally, thereby increasing confidence in strategy selection and use 
to quicken the pace and effectiveness of the implementation process. Two, they point to the subtleties interleaved in 
strategy implementation that go unnoticed or as yet we do not see. The role of common knowledge in achieving 
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collaboration among project stakeholders is one such example. Carlile (2004) argued that the key to mobilizing new 
knowledge in practice is the “capacity of the common knowledge to represent the differences and dependencies now of 
consequence and the ability of the actors involved to use it” (p. 557). When the difference between what is known and 
what is new increases, the demands on the knowledge held in common increase, and therefore the difficulties in 
working with the new knowledge, also increase. Simple transfer across practice boundaries may be easier when new 
ideas are not that distant from what practitioners already know and do, but far more difficult when novel or innovative 
thus requiring demanding new ways of thinking and doing—a subtlety we glimpsed in the requirement to create 
curriculum for intensive, tech-integrated vocabulary instruction. 

6. Limitations 

The study uses primarily descriptive statistics and qualitative methods and thus its results are limited to the site of the 
study and its participants. The number of teacher participants is small; a larger sample is needed to increase the validity 
of the strategy characteristics and patterns. The study would have been strengthened by pre/post interview data that 
probed teachers’ ideas and perceptions prior to and after the implementation process. The vocabulary intervention was 
fully implemented, i.e., across all grade levels, for less than a school year (-5 months). A strategy analysis over a full 
school year may provide a richer descriptive picture of the implementation process and/or a longitudinal study over 
time. 

7. Conclusions 

Our analytic study of the strategies used to implement vocabulary intervention instruction helped us to uncover 
descriptive details for specifying them, i.e., naming, defining and operationalizing them as they occurred in practice. 
The analysis corroborated current evidence related to the effectiveness (or lack thereof) of particular implementation 
strategies. It also revealed potential weak spots in these strategies along with emerging intricacies that may have a 
bearing on specification for purposes of replication. The analysis contributes to a better understanding of 
implementation particulars that can be considered in the design work as they strongly influence the impact of an 
intervention, i.e., what strategies work for what purpose, when and how they work, and who participates, toward 
achieving better implementation outcomes. 

References 

ATLAS.ti GmbH. (2013). Available at http://atlasti.com/ 

Azocar, F., Cuffel, B., Goldman, W., & McCarter, L. (2003). The impact of evidence-based guideline dissemination for 
the assessment and treatment of major depression in a managed behavioral health care organization. Journal of 
Behavioral Health Services Research, 30( 1), 109-118. https://doi.org/10.1007/BF02287816 

Azocar, F., Cuffel, B. D., Goldman, W., & McCulloch, J. (2001). Best practices: dissemination of guidelines for the 
treatment of major depression in a managed behavioral health care network. Psychiatric Services, 52(8), 
1014-1016. https://doi.Org/10.l 176/appi.ps.52.8.1014 

Baumann, J., Blachowicz, C., Bates, A., Cieply, C., Manyak, P., Peterson, H., & Graves, M. (2013). The development 
of a comprehensive vocabulary instruction program for nine- to eleven-year-old children using a design 
experiment approach. In T. Plomp, & N. Nieveen (Eds.), Educational design research - Part B: Illustrative cases 
(pp. 23-47). Enschede, the Netherlands: SLO. 

Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing words to life: Robust vocabulary instruction. New York, 
NY: Guilford Press. 

Biancarosa, G., Bryk A. B., & Dexter, E. (2010). Assessing the value-added effects of Literacy Collaborative 
professional development on student learning. Elementary School Journal, 111, 7-34. 

https://doi.org/10.1086/653468 

Biemiller, A., & Boote, C. (2006). An effective method for building meaning vocabulary in primary grades. Journal of 
Educational Psychology, 98(1), 44-62. https://doi.org/10.1037/0022-0663.98.E44 

Bloom, P. (2002). How children learn the meanings of words. Cambridge, MA: MIT Press. 

Bryk, A. S. (2015). Accelerating how we learn to improve. Educational Researcher, 44(9), 467-477. 
https://d 0 i. 0 rg/l 0.3102/0013189X15621543 

Butler, S., Urrutia, K., Buenger, A., Gonzalez, N., Hunt, M., & Eisenhart, C. (2010). A review of current research on 
vocabulary instruction. National Reading Technical Assistance Center. 

Carlile, P. (2004). Transferring, translating and transforming: An integrative framework for managing knowledge across 
boundaries. Organization Science, 15(5), 555-568. https://doi.org/10.1287/orsc.1040.0094 


194 



Journal of Education and Training Studies 


Vol. 5, No. 5; May 2017 


Darling-Hammond, L., Wei, R. C., Andree, A., Richardson, N., & Orphanos, S. (2009). Professional learning in the 
learning profession: A status report on teacher development in the United States and abroad. Dallas, TX: National 
Staff Development Council. Available at 

http ://www2. smcoe.k 12. ca.us/spedtf/Documents/N SDCstudyProfLeamingLearnProf.pdf 

Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation 
on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 
47(3-4), 327-350. https://doi.org/10.1007/sl0464-008-9165-0 

Edwards, A. (2012). The role of common knowledge in achieving collaboration across practices. Learning, Culture and 
Social Interaction 1, 22-32. https://doi.org/10.1016/jTcsi.2012.03.003 

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace. F. (2005). Implementation research: A 
synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health 
Institute, The National Implementation Research Network (FMHI Publication #231). 

Garet, M., Porter, A., Desimone, L., Birman, B., & Yoon, K. S. (2001). What makes professional development effective? 
Results from a national sample of teachers. American Educational Research Journal, 38(4): 915-945. 
https://doi.org/10.3102/00028312038004915 

Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M., Jones, W., & Sztejnberg, L. (2008). The impact of two 
professional development interventions on early reading instruction and achievement (NCEE 2008-4030). 
Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education 
Sciences, U.S. Department of Education. 

Glover, T. A. (Ed.). (2010). Supporting all students: The promise of Response to Intervention. In T. A. Glover & S. 
Vaughn (Eds.), The promise of Response to Intervention: Evaluating current science and practice (pp. 1-6). New 
York, NY: Guilford Press. 

Goetz, J. P., & LeCompte, M. D. (1984). Ethnography and qualitative design in educational research. FL: Academic 
Press. 

Goldenberg, C. (2004). Successful school change: Creating settings to improve teaching and learning. New York, NY: 
Teachers College Press 

Hernandez, M., & Hodges, S. (2003) Crafting logic models for systems of care: Ideas into action. [Making children’s 
mental health services successful series, volume 1]. Tampa, FL: University of South Florida, The Louis de la Parte 
Florida Mental Health Institute, Department of Child & Family Studies. 

Hosseini, Z., & Kamal, A. (2012). Developing an instrument to measure perceived technology integration knowledge of 
teachers. International Magazine on Advances in Computer Science and Telecommunications, 3(1), 79-89. 

Lesaux, N. K., Kieffer, M. J., Faller, S. E., & Kelley, J. G. (2010). The effectiveness and ease of implementation of an 
academic vocabulary intervention for linguistically diverse students in urban middle schools. Reading Research 
Quarterly, 45(2), 196-228. https://doi.Org/10.1598/RRQ.45.2.3 

Marsh, J. A., Sloan McCombs, J., Lockwood, J. R., Martorell, F., Gershwin, D., Naftel, S. & Crego, A. (2008). 
Supporting literacy across the sunshine state: A study of Florida middle school reading coaches. Santa Monica, 
CA: RAND. 

National Early Literacy Panel (2008). Developing early literacy: Report of the National Early Literacy Panel: A 
scientific synthesis of early literacy development and implications for intervention. Washington, DC: National 
Center for Family Literacy. 

National Reading Panel (2000). Report of the National Reading Panel: Teaching children to read: An evidence-based 
assessment of the scientific research literature on reading and its implications for reading instruction. Washington, 
DC: National Institute of Child Health and Human Development. 

Notion (2014). Popplet Lite. [Mobile application software]. Retrieved from 
https://itunes.apple.com/us/app/popplet-lite/id364738549?mt=8 

Nowotny, H. (2003). Democratising expertise and socially robust knowledge. Science and Public Policy , 30(3), 151-156. 
https://doi.org/10.3152/147154303781780461 

Penuel, W. R., Allen, A. R., Farrell, C., & Coburn, C. (2015). Conceptualizing research-practice partnerships as joint 
work at boundaries. Journal for Education of Students Placed at Risk (JESPAR), 20( 1-2), 182-197. 
https://doi.org/10.1080/10824669.2014.988334 


195 



Journal of Education and Training Studies 


Vol. 5, No. 5; May 2017 


Perfetti, C., & Stafura, J. (2014). Word knowledge in a theory of reading comprehension. Scientific Studies of Reading, 
18(\), 22-37. https://doi.org/10.1080/10888438.2013.827687 

Peters, D. H., Tran, N. T., & Adam, T. (2013). Implementation research in health: A practical guide. Geneva, 
Switzerland: Alliance for Health Policy and Systems Research, World Health Organization. Retrieved from 
http://who.int/alliance-hpsr/alliancehpsr_irpguide.pdf 

Pressley, M., Disney, L., & Anderson, K. (2007). Landmark vocabulary instructional research and the vocabulary 
instructional research that makes sense now. In R. K. Wagner, A. E. Muse, & K. R. Tannenbaum (Eds.), 
Vocabulary acquisition: Implications for reading comprehension (pp. 205-232). New York: Guilford Press. 

Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013). Implementation strategies: recommendations for specifying and 
reporting. Implementation Science, 8, 139. https://doi.org/10.1186/1748-5908-8-139 

Rubenstein, L. V., & Pugh, J. (2006). Strategies for promoting organizational and practice change by advancing 
implementation research. Journal of General Internal Medicine, 27(S2), S1-S70. 

https://d 0 i. 0 rg/l 0.1007/s 11606-006-0276-8 

Scarborough, H. (2001). Connecting early language and literacy to later reading (dis)abilities: Evidence, theory and 
practice. In S. B. Neuman & D. K. Dickinson (Eds.), Handbook of early literacy research (Vol. 1, pp. 97-110). 
New York: Guilford Press. 

Schofield, M. J., Edwards, K., & Pearce, R. (1997). Effectiveness of two strategies for dissemination of sun-protection 
policy in New South Wales primary and secondary schools. Australian and New Zealand Journal of Public Health, 
21(7), 743-750. https://doi.Org/10.llll/j.1467-842X.1997.tb01791.x 

Stahl, S. A. (2003). How words are learned incrementally over multiple exposures. American Educator, 27(1), 18-19, 
44. 

Storch, S. A., & Whitehurst, G. J. (2002). Oral language and code-related precursors to reading: evidence from a 
longitudinal structural model. Developmental Psychology, 28(6), 934-947. 

https://doi.Org/10.1037/0012-1649.38.6.934 

Tharp, R. G., & Gallimore, R. (1988). Rousing minds to life: Teaching, learning, and schooling in social context. 
Cambridge, UK: Cambridge University Press. 

Webster-Wright, A. (2009). Reframing professional development through understanding authentic professional learning. 
Review of Educational Research, 79(2), 702-739. https://doi.org/10.3102/0034654308330970 


Copyrights 

Copyright for this article is retained by the author(s), with first publication rights granted to the journal. 

This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license 
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly 
cited. 


196 



